Organism growth prediction system using drone-captured images

ABSTRACT

A plant growth measurement and prediction system uses drone-captured images to measure the current growth of particular plant species and/or to predict future growth of the plant species. For example, the system instructs a drone to fly along a flight path and capture images of the land below. The captured images may include both thermographic images and high-resolution images. The system processes the images to create an orthomosaic image of the land, where each pixel in the orthomosaic image is associated with a brightness temperature. The system then uses plant species to brightness temperature mappings and the orthomosaic image to identify current plant growth. The system generates a diagnostic model using the orthomosaic image to then predict future plant growth.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/798,132, entitled “ORGANISM GROWTH PREDICTION SYSTEM USINGDRONE-CAPTURED IMAGES” and filed on Oct. 30, 2017, which claims priorityunder 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No.62/515,367, entitled “ORGANISM GROWTH PREDICTION SYSTEM USINGDRONE-CAPTURED IMAGES” and filed on Jun. 5, 2017, to U.S. ProvisionalPatent Application No. 62/545,273, entitled “ORGANISM GROWTH PREDICTIONSYSTEM USING DRONE-CAPTURED IMAGES” and filed on Aug. 14, 2017, and toU.S. Provisional Patent Application No. 62/563,276, entitled “ORGANISMGROWTH PREDICTION SYSTEM USING DRONE-CAPTURED IMAGES” and filed on Sep.26, 2017, each of which is hereby incorporated by reference herein inits entirety.

BACKGROUND

Occasionally, land becomes damaged or degraded due to human actions,such as construction, contamination, the introduction of invasivespecies, and/or the like. This damage or degradation can negativelyaffect the health of native vegetation and/or the population ofendangered species. In fact, such damage or degradation can negativelyimpact humans. For example, disturbing the natural habitat of the landcan increase the risk of flooding, reduce access to clean water, orreduce recreational opportunities. Thus, a land owner or a governmententity may attempt to restore the land to its natural habitat byreintroducing native vegetation and attempting to recreate the originalnative vegetation coverage.

As part of the habitat restoration process, it may be important trackthe health and growth of the native vegetation at a site over time.Currently, this task is performed by a biologist. For example, thebiologist may visit the site, collect plant samples from a specific areaor portion of the site (e.g., along a transect), analyze the samples ina lab to identify the plant species that were present (e.g., by visuallyinspecting the features of the collected samples and comparing thosefeatures to the features of known plant species, by running a DNA testand comparing the results to the DNA of known plant species, etc.), andestimate a status of the growth of a particular plant species in theentire site based on the analysis. However, because the biologist takessamples from just a portion of the site, the plant growth estimates areoften subjective and ultimately imprecise.

SUMMARY

The systems, methods, and devices described herein each have severalaspects, no single one of which is solely responsible for its desirableattributes. Without limiting the scope of this disclosure, severalnon-limiting features will now be discussed briefly.

One aspect of the disclosure provides a system for predicting plantgrowth. The system comprises an unmanned aerial vehicle, wherein theunmanned aerial vehicle comprises a first camera and a second camera.The system further comprises a computing system comprising one or morecomputing devices, wherein the computing system is configured tocommunicate with the unmanned aerial vehicle and configured withspecific computer-executable instructions to: instruct the unmannedaerial vehicle to capture a first set of images using the first cameraand a second set of images using the second camera while flying along aflight path; receive the first set of images and the second set ofimages from the unmanned aerial vehicle; generate an orthomosaic imageusing the first set of images and the second set of images; process theorthomosaic image to identify a percentage of a land parcel that iscovered by a first plant species; generate a diagnostic model using theidentified percentage of the land parcel that is covered by the firstplant species; and predict future plant growth using the diagnosticmodel.

The system of the preceding paragraph can include any sub-combination ofthe following features: where the computing system is further configuredwith specific computer-executable instructions to: combine the first setof images according to geographic coordinates associated with each imagein the first set to form a combined first image, and orthorectify thecombined first image and the combined second image to generate theorthomosaic image; where the first camera comprises a thermal imagingcamera; where each pixel in each image in the first set corresponds to abrightness temperature, and wherein each pixel in the orthomosaic imagecorresponds to a brightness temperature; where the computing system isfurther configured with specific computer-executable instructions to:retrieve data indicating that the first plant species is associated witha first brightness temperature, process the orthomosaic image toidentify that a first set of pixels of the orthomosaic image correspondto the first brightness temperature, determine that the first set ofpixels are a first percentage of a total number of pixels in theorthomosaic image, and determine that the percentage of the land parcelthat is covered by the first plant species equals the first percentage;where the computing system is further configured with specificcomputer-executable instructions to: retrieve data indicating that thefirst plant species is associated with a first brightness temperature,process the orthomosaic image to identify that a first pixel of theorthomosaic image corresponds to the first brightness temperature, andidentify a first plant at the first pixel, wherein the first plant is anindividual plant of the first plant species; where the computing systemis further configured with specific computer-executable instructions toidentify a health of a plant at a first pixel of the orthomosaic imagebased on the brightness temperature of the first pixel; where theidentified percentage of the land parcel that is covered by the firstplant species is the percentage of the land parcel that is covered bythe first plant species at a first time, and wherein the computingsystem is further configured with specific computer-executableinstructions to: retrieve data indicating a second percentage of theland parcel that was covered by the first plant species at a second timebefore the first time, and perform a linear regression analysis on atleast one of the percentage, the second percentage, the first time, andthe second time to generate the diagnostic model; where the computingsystem is further configured with specific computer-executableinstructions to identify a predicted time when a percentage of the landparcel that is covered by the first plant species equals a desiredpercentage; where the computing system is further configured withspecific computer-executable instructions to: modify the orthomosaicimage to indicate a portion of the land parcel at which the first plantspecies needs to grow such that a percentage of the land parcel that iscovered by the first plant species equals a desired percentage, andtransmit the modified orthomosaic image to a user device; where thecomputing system is further configured with specific computer-executableinstructions to: receive flight path parameters from a user device overa network, and instruct the unmanned aerial vehicle to capture the firstset of images using the first camera and the second set of images usingthe second camera while flying along a flight path in a manner definedby the flight path parameters; and where the flight path parameterscomprise at least one of geographic coordinates, waypoints, flightlength, flight time, speed, altitude, camera shooting angle, cameracapture mode, or camera resolution.

Another aspect of the disclosure provides a computer-implemented methodof predicting plant growth. The computer-implemented method comprises,as implemented by one or more computing devices configured with specificcomputer-executable instructions: instructing an aerial vehicle tocommence a flight such that the aerial vehicle captures a first set ofimages using a first camera and captures a second set of images using asecond camera; receiving the first set of images and the second set ofimages from the aerial vehicle; generating an orthomosaic image usingthe first set of images and the second set of images; processing theorthomosaic image to identify a percentage of a land parcel that iscovered by a first plant species; generating a diagnostic model usingthe identified percentage of the land parcel that is covered by thefirst plant species; and predicting future plant growth using thediagnostic model.

The computer-implemented method of the preceding paragraph can includeany sub-combination of the following features: where generating anorthomosaic image using the first set of images and the second set ofimages further comprises: combining the first set of images according togeographic coordinates associated with each image in the first set toform a combined first image, combining the second set of imagesaccording to geographic coordinates associated with each image in thesecond set to form a combined second image, and orthorectifying thecombined first image and the combined second image to generate theorthomosaic image; where the first camera comprises a thermal imagingcamera; where each pixel in each image in the first set corresponds to abrightness temperature, and wherein each pixel in the orthomosaic imagecorresponds to a brightness temperature; where processing theorthomosaic image to identify a percentage of a land parcel that iscovered by a first plant species further comprises: retrieving dataindicating that the first plant species is associated with a firstbrightness temperature, processing the orthomosaic image to identifythat a first set of pixels of the orthomosaic image correspond to thefirst brightness temperature, determining that the first set of pixelsare a first percentage of a total number of pixels in the orthomosaicimage, and determining that the percentage of the land parcel that iscovered by the first plant species equals the first percentage; wherethe computer-implemented method further comprises: retrieving dataindicating that the first plant species is associated with a firstbrightness temperature, processing the orthomosaic image to identifythat a first pixel of the orthomosaic image corresponds to the firstbrightness temperature, and identifying a first plant at the firstpixel, wherein the first plant is an individual plant of the first plantspecies; where the computer-implemented method further comprisesidentifying a health of a plant at a first pixel of the orthomosaicimage based on the brightness temperature of the first pixel; and wherethe identified percentage of the land parcel that is covered by thefirst plant species is the percentage of the land parcel that is coveredby the first plant species at a first time, and wherein generating adiagnostic model using the identified percentage of the land parcel thatis covered by the first plant species further comprises: retrieving dataindicating a second percentage of the land parcel that was covered bythe first plant species at a second time before the first time, andperforming a linear regression analysis on at least one of thepercentage, the second percentage, the first time, and the second timeto generate the diagnostic model.

Another aspect of the disclosure provides non-transitory,computer-readable storage media comprising computer-executableinstructions for predicting plant growth, wherein thecomputer-executable instructions, when executed by a computer system,cause the computer system to: instruct an aerial vehicle to commence aflight such that the aerial vehicle captures a first set of images usinga first camera; process the first set of images received from the aerialvehicle; generate an orthomosaic image using the first set of images;process the orthomosaic image to identify a percentage of a land parcelthat is covered by a first plant species; generate a diagnostic modelusing the identified percentage of the land parcel that is covered bythe first plant species; and predict future plant growth using thediagnostic model.

The non-transitory, computer-readable storage media of the precedingparagraph can include any sub-combination of the following features:where the first camera comprises a thermal imaging camera, wherein eachpixel in each image in the first set corresponds to a brightnesstemperature, and wherein each pixel in the orthomosaic image correspondsto a brightness temperature; where the computer-executable instructionsfurther cause the computer system to: retrieve data indicating that thefirst plant species is associated with a first brightness temperature,process the orthomosaic image to identify that a first set of pixels ofthe orthomosaic image correspond to the first brightness temperature,determine that the first set of pixels are a first percentage of a totalnumber of pixels in the orthomosaic image, and determine that thepercentage of the land parcel that is covered by the first plant speciesequals the first percentage; and where the identified percentage of theland parcel that is covered by the first plant species is the percentageof the land parcel that is covered by the first plant species at a firsttime, and wherein the computer-executable instructions further cause thecomputer system to: retrieve data indicating a second percentage of theland parcel that was covered by the first plant species at a second timebefore the first time, and perform a linear regression analysis on atleast one of the percentage, the second percentage, the first time, andthe second time to generate the diagnostic model.

Another aspect of the disclosure provides a system for detecting planthealth. The system comprises an unmanned aerial vehicle, wherein theunmanned aerial vehicle comprises a camera. The system further comprisesa computing system comprising one or more computing devices, wherein thecomputing system is configured to communicate with the unmanned aerialvehicle and configured with specific computer-executable instructionsto: instruct the unmanned aerial vehicle to capture a first set ofimages using the camera while flying along a flight path; receive thefirst set of images from the unmanned aerial vehicle; for each image inthe first set of images, convert the respective image into a planthealth image; process the plant health images; and transmit a message toan external system to cause an action to be performed based on theprocessing of the plant health images.

The system of the preceding paragraph can include any sub-combination ofthe following features: where the external system comprises one of anirrigation system or a lighting system; and where the action comprisesone of a lighting adjustment, a lighting schedule adjustment, a wateringadjustment, a watering schedule adjustment, or a notification thatplants need to be pruned.

BRIEF DESCRIPTION OF DRAWINGS

Throughout the drawings, reference numbers may be re-used to indicatecorrespondence between referenced elements. The drawings are provided toillustrate example embodiments described herein and are not intended tolimit the scope of the disclosure.

FIG. 1 is a block diagram of an illustrative operating environment inwhich a plant growth prediction system uses images captured by an aerialvehicle to determine current and/or predict future plant growth.

FIG. 2 is a flow diagram illustrating the operations performed by thecomponents of the operating environment of FIG. 1 to generate anorthomosaic image for an initial flight, according to one embodiment.

FIGS. 3A-3B are flow diagrams illustrating the operations performed bythe components of the operating environment of FIG. 1 to predict plantgrowth after a flight that follows the initial flight, according to oneembodiment.

FIG. 4 is a flow diagram depicting a plant growth prediction routineillustratively implemented by a plant growth prediction system,according to one embodiment.

FIG. 5A illustrates a user interface displaying a site and a list ofbasic flight path parameters.

FIG. 5B illustrates a user interface displaying the site and a list ofadvanced flight path parameters.

FIG. 6A illustrates a user interface displaying plant health of a siteoverlaid over a high-resolution image of the site depicted in a window.

FIG. 6B illustrates a user interface displaying elevation of a siteoverlaid over a high-resolution image of the site depicted in thewindow.

FIG. 7 illustrates a user interface 700 displaying individuallyidentified plants in a site overlaid over a high-resolution image of thesite depicted in a window.

FIGS. 8A-8B illustrate a user interface 800 displaying tools foranalyzing plants at a site depicted in a window 810.

FIG. 9 is a flow diagram depicting a plant health prediction routine 900illustratively implemented by a plant growth prediction system,according to one embodiment.

FIG. 10 illustrates an exemplary aerial vehicle, such as the aerialvehicle of FIG. 1.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

As described above, the current process of estimating plant growth at asite (e.g., a parcel or plot of land) is subjective and imprecise. Forexample, portions of a site may be inaccessible by a human due to theterrain, plant species that are present, and/or animal species that arepresent. In some cases, portions of a site may be accessed. However, ifsuch portions are accessed, this may cause a disturbance in the site andresult in inaccurate measurements and/or damage to the native vegetationand/or animal species. Thus, biologists often take samples from just atransect of the site and generate plant growth estimates based on anextrapolation of the data collected from the transect. Conventionalcomputing systems could be used to generate plant growth estimates basedon samples collected by biologists. However, these conventionalcomputing systems do not resolve inaccuracies that may result from thesamples being collected in just a transect of a site.

In particular, a biologist may analyze the collected samples to identifythe plant species that were present in the transact (e.g., by visuallyinspecting the features of the collected samples and comparing thosefeatures to the features of known plant species, by running a DNA testand comparing the results to the DNA of known plant species, etc.) andestimate a status of the growth of a particular plant species in theentire site based on the analysis. However, a site generally will nothave a uniform distribution of plant species. For example, the terrainof the site, the amount of available sunshine, the level of the watertable, and/or other factors may affect whether a particular plantspecies will be present in a specific transect of the site. Thus, theestimation then may greatly depend on the transect from which thebiologist collected samples. Given the subjectivity and variability ofthis estimation, it may be important to implement techniques foranalyzing the plant growth across an entire site.

Accordingly, aspects of the present disclosure provide a plant growthmeasurement and prediction system that uses drone-captured images tomeasure the current growth of particular plant species and/or to predictfuture growth of the plant species. For example, a user, via a userdevice, may communicate with the plant growth prediction system to setthe flight path of a drone or another aerial vehicle. The drone may beequipped with one or more cameras, such as a thermal imaging camera, ahigh-resolution camera (e.g., 4K, 8K, etc.), and/or the like. In anembodiment, the flight path is set such that the drone will captureimages covering an entire site. Once the flight path is set, the plantgrowth prediction system can transmit flight path data to the drone overa network (or a wired or wireless point-to-point link) and instruct thedrone to conduct a series of flights over a period of time, where eachflight follows the same flight path. For example, flights may take placeonce ever few weeks, months, years etc. The drone may communicate with asatellite system (e.g., a global positioning system (GPS)) and/orterrestrial system to fly according to the received flight path data. Asthe drone flies along the provided flight path, the drone mayperiodically capture images of the land underneath the drone. Forexample, the drone can capture images directly underneath the drone(e.g., the camera(s) may be positioned such that a lens is approximatelyparallel with the land) and/or at an angle (e.g., the camera(s) may bepositioned such that a lens deviates from being parallel with the landby a certain angle). The drone may capture images using one or more ofthe cameras and transmit such images to the plant growth predictionsystem in real-time and/or after the flight is complete.

The plant growth prediction system may stitch the received imagestogether to form a single stitched image. For example, each image (e.g.,the boundaries of the image, edges of the image, vertices of the image,individual pixels within the image, etc.) may correspond with one ormore geographic coordinates (e.g., GPS coordinates). The plant growthprediction system can stitch the received images using the geographiccoordinates as a guide (e.g., the plant growth prediction system canappend an edge of one image to a portion of another image if the edgeand portion each correspond to the same geographic coordinate or rangeof geographic coordinates). As described above, the flight path may beset such that the drone captures images covering an entire site. Thus,the stitched image may be an image that captures an entire site. If thedrone includes different types of cameras, the plant growth predictionsystem can segregate images corresponding to a particular type of cameraand stitch together those images that originated from a particular typeof camera. Thus, the plant growth prediction system may form multiplestitched images. The plant growth prediction system can then combine themultiple stitched images to form an orthomosaic image.

As an illustrative example, if the drone captured both thermographicimages (e.g., using a thermal camera) and high-resolution images (e.g.,using a high-resolution camera), then the plant growth prediction systemcan generate a thermographic stitched image by stitching thethermographic images and a high-resolution stitched image by stitchingthe high-resolution images. Given that thermal cameras generally producean image that identifies a brightness temperature of various objectscaptured within the image, the thermographic stitched image may then bean image that identifies the brightness temperature of various objectscaptured within each of the stitched thermographic images. The plantgrowth prediction system can then process the two stitched images byoverlaying the thermographic stitched image over the high-resolutionstitched image and identifying, for each pixel in the high-resolutionstitched image, a brightness temperature level. The result of processingthe stitched images may be an orthomosaic image.

Note that while the distribution of pixels within an image describes thespatial structure of the image, the radiometric characteristics of thepixels describe the actual content depicted in the image. For example,the sensitivity of a film or a sensor of the camera 132 to the magnitudeof the electromagnetic energy present in the environment may determinethe radiometric resolution of the film or sensor. The radiometricresolution of the film or sensor may describe the film or sensor'sability to discriminate very slight differences in the electromagneticenergy present in the environment. As an illustrative example, thehigher the radiometric resolution of a sensor, the more sensitive thesensor is to detecting small differences in the intensity orreflectivity of the electromagnetic energy. Thus, the values of thebrightness temperatures depicted within the thermographic images and thedifferences in brightness temperature values between different pixelsmay depend on the radiometric resolution of the film or sensor of thecamera 132 that captured the thermographic images.

The first time a drone captures images for a particular site, theresulting orthomosaic image may be stored for future use. Once one ormore additional flights take place, the plant growth prediction systemcan form orthomosaic images for each of these additional flights and usethe orthomosaic images along with the initial orthomosaic image toidentify current plant growth and/or predict future plant growth.

For example, given a known set of conditions, a plant species mayradiate its kinetic temperature at a certain brightness temperature.Thus, a mapping between brightness temperature and plant species can begenerated and stored in a data store. In addition, other organic andinorganic matter, such as animals, organisms other than plants andanimals, dirt, water, etc., may radiate their kinetic temperature atcertain brightness temperatures and mappings can be generated and storedfor these types of matter as well. The plant growth prediction systemcan retrieve the mappings for processing each orthomosaic image. Inparticular, the plant growth prediction can, for each pixel in eachorthomosaic image, use the mappings to identify a plant species (oranimal species, other organisms, dirt, water, etc.) that maps to thebrightness temperature of the respective pixel. A user, via the userdevice, may have specified certain plant species that are of interest.For example, in the context of habitat restoration, the user may beinterested in monitoring the growth of a native plant species at a site.In addition, the user may be interested in monitoring the amount of fillpresent at the site (e.g., open space, dirt areas, and/or other areasthat were damaged or regraded and need to be filled in), the growth ofinvasive species (e.g., weeds, non-native animals, and/or other objectsblown in from surrounding areas), and/or plant diversity (e.g., thenumber of different types of plant species that are present in thesite). The plant growth prediction system can then, for each orthomosaicimage, use the identified plant species to determine, for a timecorresponding to the respective orthomosaic image, a percentage of thesite that is covered by the native species, a percentage of the sitethat is covered by fill, a percentage of the site that is covered by aninvasive species, a count representing the plant diversity in the site,and/or the like. A percentage of a site that is covered by a particularobject is referred to herein as a “coverage percentage.”

Using the coverage percentages determined for a native species over aperiod of time, the plant growth prediction system can generate adiagnostic model. The diagnostic model can be used to predict futurenative species growth (e.g., represented by a coverage percentage). Forexample, the plant growth prediction system can perform a linearregression analysis of the coverage percentages, a cubic polynomialregression analysis of the coverage percentages, and/or the like togenerate the diagnostic model. The plant growth prediction system canfurther generate a diagnostic model for fill, an invasive species,and/or plant diversity using the same techniques.

The diagnostic model(s) may each output a coverage percentage or plantdiversity count as a function of time. Thus, the plant growth predictionsystem may then use the diagnostic model(s) to predict future coveragepercentages and/or a plant diversity count at various times in thefuture. For example, the plant growth prediction system can use thediagnostic model corresponding to a native species to predict a time atwhich the native species will have a coverage percentage correspondingto a desired coverage percentage (e.g., a coverage percentage thatindicates that the habitat restoration is complete). The plant growthprediction system can package the predictions into a report and transmitthe report to a user device. In addition, the plant growth predictionsystem can modify one or more orthomosaic images to indicate certaininformation. For example, the plant growth prediction system canannotate a latest orthomosaic image to indicate areas where a nativespecies is growing and areas in which the native species needs to grow(e.g., in fill areas) to meet a desired coverage percentage. The plantgrowth prediction system can also send the modified orthomosaic image(s)to the user device.

Thus, unlike the subjective estimations performed by a biologist, thetechniques implemented by the plant growth prediction system describedherein can result in an objective analysis of current and future plantgrowth. For example, a drone or other aerial vehicle can reach areas ofa site that otherwise may be inaccessible or should not be accessed dueto the potential damage that may be incurred. Thus, the drone canmeasure data for an entire site, rather than just a transect, andprovide such data to the plant growth prediction system. The plantgrowth prediction system then can use the data for an entire site todetermine an accurate representation of the current plant growth andpredict future plant growth. The plant growth prediction system,therefore, determines a more accurate representation of the currentplant growth and/or predicts future plant growth using techniques thatpreviously could not even be performed by biologists.

In addition, because the drone is not necessarily capturing physicalsamples for later analysis (although this may occur in certainembodiments, as described below), the plant growth prediction systemimplements different techniques and rules than a biologist inidentifying the presence of plant species for the purpose of determiningcurrent plant growth. For example, while a biologist may visit a site,retrieve physical vegetation samples, and conduct tests on the physicalvegetation samples in a lab, the plant growth prediction system insteadcontrols a drone and implements image processing techniques to identifycertain characteristics of objects present in the drone-captured images(e.g., brightness temperature). Such image processing techniques includestitching images, merging stitched images, identifying the brightnesstemperature of a pixel in the merged image, and comparing the identifiedbrightness temperature to plant species-brightness temperature mappingsto determine a plant species that is present at the pixel. In fact, abiologist would not even be able to perform these image processingtechniques (e.g., by visually inspecting the physical vegetationsamples) because the brightness temperatures analyzed by the plantgrowth prediction system are derived from drone-captured thermal images,which are images depicting light that is invisible to humans (e.g.,infrared light).

Accordingly, the plant growth prediction system described hereinprovides an improvement in computer-related technology (e.g., byallowing computing devices to produce more accurate determinations ofcurrent plant growth at a site and/or more accurate predictions offuture plant growth at the site) using specific techniques that are notused and cannot be used by humans, who instead rely on subjectivedeterminations in estimating current plant growth.

While the primary use case for the plant growth prediction systemdescribed herein is monitoring of habitat restoration at a site, this ismerely for illustrative purposes and is not meant to be limiting. Giventhat organic and inorganic matter produce a certain brightnesstemperature under known conditions, the techniques described herein asbeing implemented by the plant growth prediction system can be appliedto other diverse use cases. For example, the techniques described hereincan be implemented for inspecting golf courses (e.g., determiningcurrent and/or predicting future plant growth), water management (e.g.,the brightness temperature of soil may change as the water table risesand falls, so the techniques described herein can be used to evaluatewater table levels; evaluating the surface area covered by a body ofwater over time for flood management purposes, for irrigation purposes,etc.; etc.), inspecting trees (e.g., determining current levels ofand/or predicting future levels of the amount of cover or shade providedby trees, determining current and/or predicting future tree health giventhat the brightness temperature changes as trees dry out and/or die,etc.), inspect plant health (e.g., determining current levels of and/orpredicting future levels of plant health given that the brightnesstemperature changes as plants become sick or healthier), monitoringanimals (e.g., determining current and/or predicting future bird counts,determining current and/or predicting future endangered species counts,etc.), monitoring invasive species (e.g., determining current and/orpredicting future weed growth), mapping fire fuel (e.g., determiningcurrent and/or predicting future growth of plants susceptible toextending the life of a fire), inspecting erosion (e.g., different soillayers may correspond to different brightness temperatures, so themovement or appearance of soil layers over time can be determined and/orpredicted), evaluating common areas (e.g., for a homeowners associationor park to determine and/or predict plant growth), inspecting miningoperations (e.g., determining current and/or predicting future watermovement, determining current and/or predicting future growth ofreintroduced plants, etc.), landscaping (e.g., determining currentand/or predicting future plant growth), monitoring a waste reclamationsite (determining current and/or predicting future plant growth),monitoring vineyards (e.g., determining current and/or predicting futuregrapevine growth, determining current and/or predicting future invasiveplant and/or animal species growth, etc.), monitoring nurseries(determining current and/or predicting future plant growth), and/or forany other use cases in which it may be beneficial to measure and/orpredict plant growth.

The foregoing aspects and many of the attendant advantages of thisdisclosure will become more readily appreciated as the same becomebetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings.

Example Plant Growth Prediction Environment

FIG. 1 is a block diagram of an illustrative operating environment 100in which a plant growth prediction system 120 uses images captured by anaerial vehicle 130 to determine current plant growth and/or predictfuture plant growth. The operating environment 100 further includesvarious user devices 102 that may communicate with the plant growthprediction system 120 via a network 110 to provide flight pathparameters, to receive generated reports, and/or to receive modifiedimages indicating current and/or predicted plant growth. In addition,the operating environment 100 may further include an irrigation system150 and a water system 160 that can be controlled directly or indirectlyby the plant growth prediction system 120.

The aerial vehicle 130 may be an unmanned aerial vehicle. For example,the aerial vehicle 130 may be four or six rotor drone. Alternatively,the aerial vehicle 130 can be a manned aerial vehicle. In someembodiments, two or more unmanned aerial vehicles may be usedconcurrently at a given site to perform the functionality describedherein with respect to a single aerial vehicle 130. The aerial vehicle130 may include one or more cameras 132. For example, each camera 132may be a different type of camera, such as a thermal imaging camera, ahigh-resolution camera (e.g., 4K, 8K, etc.), and/or the like.Alternatively or in addition, one camera 132 may include multiple lensessuch that the camera 132 can capture different types of images. Thecamera(s) 132 may be mounted to a bottom and/or side surface of theaerial vehicle 130 such that the camera(s) 132 can capture images of theland underneath the aerial vehicle 130. In some embodiments, one or moreof the cameras 132 are mounted to a gimbal that mounts to a bottomand/or side surface of the aerial vehicle 130 to allow for camera 132rotation. One or more of the cameras 132 may include a network interface(e.g., a universal serial bus (USB) port, an Ethernet port, a wirelesstransceiver, etc.) to communicate with the plant growth predictionsystem 120 via the network 110 (or via a wired or wirelesspoint-to-point link). Alternatively, the cameras 132 may transmit data(e.g., captured images) to a network interface (e.g., a universal serialbus (USB) port, an Ethernet port, a wireless transceiver, etc., notshown) of the aerial vehicle 130 for eventual transmission to the plantgrowth prediction system 120.

The aerial vehicle 130 may further include a flight path controller 138.The flight path controller 138 may communicate with the plant growthprediction system 120 to receive flight path parameters. For example,flight path parameters may include a flight path (e.g., one or moregeographic coordinates, waypoints, flight length, flight time, speed,altitude, flight course mode, a front overlap ratio (e.g., a frontoverlap of the defined boundaries of a capture site that is necessaryfor the one or more cameras 132 to fully capture the capture site,represented as a percentage), a side overlap ratio (e.g., a side overlapof the defined boundaries of a capture site that is necessary for theone or more cameras 132 to fully capture the capture site, representedas a percentage), a course angle, etc.), a shooting angle (e.g., anangle at which one or more cameras 132 is positioned to capture images),a capture mode (e.g., a setting indicating when the one or more cameras132 capture images), a gimbal pitch angle (e.g., an angle of a lens ofthe one or more cameras 132), an end-mission action (e.g., hover, returnto start, etc.), camera resolution, and/or the like. Alternatively, theflight path controller 138 can communicate directly with a user device102, such as a user device 102 present at a site with the aerial vehicle130.

Upon receiving flight path parameters, the flight path controller 138can control the operation of the aerial vehicle 130 according to theflight path parameters. For example, the flight path controller 138 cantransmit instructions to various components of the aerial vehicle 130 tocause the aerial vehicle 130 to take off from a current location, followa certain flight path, instruct the camera(s) 132 to capture images atthe appropriate time and at the appropriate angle, and land once theflight is complete. Commercially available drones, such as the DJIPHANTOM 3 or INSPIRE 1 PRO unmanned aerial vehicles, and associated code(e.g., such as the live view application provided with the DJI PHANTOM 3unmanned aerial vehicle) may provide such features. In some embodiments,the plant growth prediction system 120 can transmit updated flight pathparameters to the flight path controller 138 while the aerial vehicle130 is in flight. When updated flight path parameters are received inflight, the flight path controller 138 can transmit instructions tovarious components of the aerial vehicle 130 to cause the aerial vehicle130 to adjust flight according to the updated flight path parameters.

The flight path controller 138 may further include instructions that,when executed, cause the aerial vehicle 130 to deviate from the selectedflight path at the instruction of a user and/or automatically. Forexample, as described below, the aerial vehicle 130 can transmitcaptured images in real-time (e.g., as the images are captured) to theplant growth prediction system 120. The plant growth prediction system120 (e.g., the image processor 122 described below) may provide one ormore user devices 102 with access to the captured images as they arereceived during a flight. For example, the plant growth predictionsystem 120 (e.g., the user interface generator 131 described below) maygenerate user interface data that is transmitted to a user device 102and that causes the user device 102 to display a user interface showingthe images as the images are captured by the one or more cameras 132. Auser viewing the user interface and captured images may notice an objectof interest and can use controls provided by the user interface totransmit instructions to the aerial vehicle 130 via the plant growthprediction system 120 that causes the aerial vehicle 130 to return tothe location where the object of interest was noticed. As anotherexample, the plant growth prediction system 120 (e.g., the imageprocessor 122) and/or the flight path controller 138 can be configuredto process captured images as the camera(s) 132 captures such images toidentify certain objects and, if such objects are identified, instructthe aerial vehicle 130 or otherwise cause the aerial vehicle 130 todeviate from the flight path to revisit the identified object (e.g., tocapture additional images). The plant growth prediction system 120and/or flight path controller 138 can use data indicating the shapeand/or brightness temperature of specific objects to process a capturedimage and determine whether an object with the same shape and/orbrightness temperature is present. Note that the flight path controller138 may intercept and/or receive images captured by the camera(s) 132 inorder to perform the processing.

Optionally, the aerial vehicle 130 includes a speaker 134 and/ormicrophone 136. For example, the aerial vehicle 130 may be instructed tocapture images for the purpose of monitoring an animal population (e.g.,birds, rodents, deer, endangered species, etc.). The speaker 134 mayoutput a sound that resembles a sound produced by the subject animalspecies (e.g., a bird call). The microphone 136 may be enabled to listenfor sounds that are produced in response to the sound output by thespeaker 134. The aerial vehicle 130 can transmit the sounds picked up bythe microphone 136 along with the geographic coordinates at which thesounds were received to the plant growth prediction system 120 foranalysis. In particular, the plant growth prediction system 120 maystore a mapping of sounds to specific animal species. Thus, the plantgrowth prediction system 120 can process the received sounds to identifywhether the sounds resemble sounds associated with a known animalspecies. If a match occurs, then the plant growth prediction system 120can determine that at least one animal of the animal species was presentin the vicinity of a location at which the sound was picked up by themicrophone 136. This audio processing can supplement the imageprocessing described herein to provide a more accurate determination ofa current animal population and/or a more accurate prediction of afuture animal population. In addition, the audio data may help the plantgrowth prediction system 120 provide accurate determinations of acurrent animal population and/or accurate predictions of a future animalpopulation even if the particular animal species is not visible in thecaptured images.

In further embodiments, not shown, the aerial vehicle 130 includes amechanical and/or pneumatic attachment (e.g., a mechanical and/orpneumatic arm) configured to obtain and hold items, collect samples,and/or the like. During flight, the aerial vehicle 130 can use themechanical attachment to perform such actions and record, using thecurrent geographic coordinates of the aerial vehicle 130, a location atwhich such actions were performed. The location information may then beused in determining current and/or predicted plant growth and/or formodifying an orthomosaic image to indicate a location where an actionwas performed.

In further embodiments, the aerial vehicle 130 may include sensors, notshown, to perform obstacle avoidance. For example, the aerial vehicle130 may be flying at a low altitude (e.g., 8-9 meters). Tree branches,terrain, and/or other objects may therefore impede the flight path ofthe aerial vehicle 130. The aerial vehicles 130 (e.g., the flight pathcontroller 138) can therefore use the sensors to detect objects to thefront and/or side of the aerial vehicle 130, adjust a flight path of theaerial vehicle 130 to avoid the detected objects, and then return to theflight path set by the flight path parameters once the aerial vehicle130 is clear of the detected objects.

The plant growth prediction system 120 can be a computing systemconfigured to periodically instruct the aerial vehicle 130 to captureimages along a flight path above a site and use the captured images todetermine current plant growth and/or predict future plant growth at thesite. For example, the plant growth prediction system 120 may be asingle computing device, or it may include multiple distinct computingdevices, such as computer servers, logically or physically groupedtogether to collectively operate as a server system, or independentcomponents or devices that are or are not networked together, but thatare used in combination to perform the operations described herein. Asan illustrative example, one computing device in the plant growthprediction system 120 may perform the operations described below withrespect to aerial vehicle controller 121, while another, separatecomputing device in the plant growth prediction system 120 may performthe operations described below with respect to the plant growthpredictor 124. The components of the plant growth prediction system 120can each be implemented in application-specific hardware (e.g., a servercomputing device with one or more ASICs) such that no software isnecessary, or as a combination of hardware and software. In addition,the modules and components of the plant growth prediction system 120 canbe combined on one server computing device or separated individually orinto groups on several server computing devices. In some embodiments,the plant growth prediction system 120 may include additional or fewercomponents than illustrated in FIG. 1.

In some embodiments, the features and services provided by the plantgrowth prediction system 120 may be implemented as web servicesconsumable via the communication network 110. In further embodiments,the plant growth prediction system 120 is provided by one more virtualmachines implemented in a hosted computing environment. The hostedcomputing environment may include one or more rapidly provisioned andreleased computing resources, which computing resources may includecomputing, networking and/or storage devices. A hosted computingenvironment may also be referred to as a cloud computing environment.

The plant growth prediction system 120 may include various modules,components, data stores, and/or the like to provide the plant growthmeasurement and prediction functionality described herein. For example,the plant growth prediction system 120 may include an aerial vehiclecontroller 121, an image processor 122, a diagnostic model generator123, a plant growth predictor 124, a flight path data store 125, animage data store 126, a plant profile data store 127, a model data store128, a coverage percentage data store 129, and a user interfacegenerator 131.

The aerial vehicle controller 121 may receive flight path parametersfrom the user device 102 via the network 110. In an embodiment, the userdevice 102 sets the flight path such that the aerial vehicle 130captures images covering an entire site. As described in greater detailbelow with respect to FIGS. 5A-5B, the user device 102 may present auser interface that allows a user to visually set the flight path andone or more flight path parameters. The user device 102 may additionalprovide the aerial vehicle controller 121 with a set of times or phasesat which the aerial vehicle 130 should conduct flights. For example, theuser device 102 may indicate that a first flight should occur before anyimpact to the site has occurred, a second flight should occur onceimpact to the site has commenced (e.g., a portion of the site is underconstruction, plant material has been removed, surface lines have beeninstalled, trenches have been dug, etc.), a third flight should occuronce plants are being installed, a fourth flight should occur as plantmaterial begins to mature, and/or any times after the plant material hasbegun to mature. Time intervals between flights may be in the minutes,hours, days, weeks, months, years, etc. In an embodiment, the aerialvehicle controller 121 stores the flight path parameters in the flightpath data store 125 in an entry associated with the site such that theflight path parameters can be retrieved and reused for each subsequentflight (e.g., each flight may occur according to the same flight pathparameters).

Once the aerial vehicle controller 121 determines that the aerialvehicle 130 should conduct a flight at a current time, a project membermay bring the aerial vehicle 130 to the site (e.g., based on a reminderprovided by the plant growth prediction system 120). The aerial vehiclecontroller 121 can transmit the flight path parameters to the aerialvehicle 130 over the network 110 and instruct the aerial vehicle 130(e.g., the flight path controller 138) to begin the flight. The aerialvehicle 130 (e.g., the flight path controller 138) may communicate witha satellite system (e.g., a GPS system) and/or terrestrial system to flyaccording to the received flight path parameters. As the aerial vehicle130 travels along the indicated flight path, the aerial vehicle 130captures images of the land underneath the aerial vehicle 130 at aninterval determined by the capture mode and in a manner determined bythe shooting angle and/or the gimbal pitch angle using the one or morecameras 132. For example, the one or more cameras 132 can capture imagesdirectly underneath the aerial vehicle 130 (e.g., the camera(s) 132 maybe positioned such that a lens is approximately parallel with the land,facing straight down) and/or at an angle (e.g., the camera(s) 132 may bepositioned such that a lens deviates from being parallel with the landby a certain angle). The camera(s) 132 and/or a network interface (notshown) may transmit captured images to the image processor 122 inreal-time (e.g., as the images are captured) and/or after the flight iscomplete.

The image processor 122 may stitch the received images together to forma single stitched image. For example, the aerial vehicle 130 maytransmit metadata associated with each image. The metadata may indicateportions of the image (e.g., the boundaries of the image, edges of theimage, vertices of the image, individual pixels within the image, etc.)that correspond to particular geographic coordinates (e.g., asdetermined by the aerial vehicle 130 via communications with the GPSsystem). The image processor 122 can stitch the received images usingthe geographic coordinates provided in the metadata as a guide. Forexample, the image processor 122 can append an edge of one image to anedge of another image if the edges each correspond to the samegeographic coordinate or range of geographic coordinates. As anotherexample, the image processor 122 can append an edge of one image to aportion of another image if the edge and portion each correspond to thesame geographic coordinate or range of geographic coordinates. Asdescribed above, the flight path may be set such that the aerial vehicle130 captures images covering an entire site. Thus, the stitched imagemay be an image that captures an entire site.

If the aerial vehicle 130 includes different types of cameras 132, thenthe image processor 122 can segregate images corresponding to aparticular type of camera 132 and stitch together those images thatoriginated from a particular type of camera 132. Thus, the imageprocessor 132 may form multiple stitched images. The image processor 122can then combine the multiple stitched images to form an orthomosaicimage. For example, the image processor 122 may use a digital elevationmodel (DEM) of the site to combine the multiple stitched images usingorthorectification techniques such that the stitched images aregeometrically corrected to have a uniform scale. In other words, theimage processor 122 may form an orthomosaic image that has the same lackof distortion as a map and can be used to measure true distances.

As an illustrative example, if the aerial vehicle 130 captures boththermographic images (e.g., using a thermal camera 132) andhigh-resolution images (e.g., using a high-resolution camera 132), thenthe image processor 122 can generate a thermographic stitched image bystitching the thermographic images and a high-resolution stitched imageby stitching the high-resolution images. Given that thermal cameras 132generally produce an image that identifies a brightness temperature ofvarious objects captured within the image, the thermographic stitchedimage may then be an image that identifies the brightness temperature ofvarious objects captured within each of the stitched thermographicimages. The image processor 122 can then process the two stitched imagesby overlaying the thermographic stitched image over the high-resolutionstitched image and identifying, for each pixel in the high-resolutionstitched image, a brightness temperature level. The result of processingthe stitched images may be an orthomosaic image.

Given that different objects have different emissivity levels, the imageprocessor 122 may adjust an emissivity sensitivity level of one or moreof the thermographic images before the stitching is performed dependingon the type of prediction that the plant growth prediction system 120will eventually perform. For example, plant material may have highemissivity levels (e.g., between 0.94 and 0.96), whereas water may havelower emissivity levels (e.g., around 0.67). Thus, as an illustrativeexample, the image processor 122 may adjust the emissivity sensitivitylevel of the thermographic images to between 0.94 and 0.96 ifdetermining current and/or predicting future plant growth and may adjustthe emissivity sensitivity level of the thermographic images to around0.67 if determining current and/or predicting future water table levels.

The pixels in an orthomosaic image may be labeled, shaded (e.g., withspecific colors that indicate brightness temperature), or otherwiseannotated to indicate the identified brightness temperature levels. Suchlabels, shading, or annotations may be visible or invisible when adevice, such as the user device 102, displays the orthomosaic image. Forexample, a background of the orthomosaic image may be thegeometrically-corrected high-resolution stitched image. The pixels ofthe orthomosaic image may then be shaded colors corresponding to theidentified brightness temperature of the respective pixels.Alternatively, the orthomosaic image may be associated with metadatathat identifies the brightness temperature levels of each pixel in theorthomosaic image.

The image processor 122 may store the orthomosaic image generated fromimages captured during a flight in the image data store 126. If a storedorthomosaic image corresponds to a first flight for a particular site,the plant growth prediction system 120 may take no further action.However, once one or more additional flights take place and the imageprocessor 122 forms one or more additional orthomosaic images, the plantgrowth prediction system 120 can use the stored orthomosaic images toidentify current plant growth and/or predict future plant growth at thesite.

For example, given a known set of conditions, a plant species mayradiate its kinetic temperature at a certain brightness temperature.Thus, mappings between brightness temperature and plant species can begenerated and stored in the plant profile data store 127. In addition,other organic and inorganic matter, such as animals, organisms otherthan plants and animals, dirt, water, etc., may radiate their kinetictemperature at certain brightness temperatures and mappings can begenerated and stored in the plant profile data store 127 for these typesof matter as well.

In further embodiments, leaf, plant, and/or animal shape (e.g., detectedvia a pattern recognition process implemented by the image processor122); leaf, plant, and/or animal color; leaf, plant, and/or animal size;and/or the like may be mapped to particular plant and/or animal species.Such mappings can also be stored in the plant profile data store 127 foruse by the image processor 122 identifying plant and/or animal species.

The image processor 122 can retrieve one or more mappings from the plantprofile data store 127 for individually processing each orthomosaicimage. In particular, the image processor 122 can, for each pixel ineach orthomosaic image, use the mappings to identify a plant species (oranimal species, other organisms, dirt, water, etc.) that maps to thebrightness temperature of the respective pixel.

Optionally, the image processor 122 can, for some or all of the pixelsin each orthomosaic image, use the mappings to identify individualplants (or animals, other organisms, dirt, water, etc.). For example, auser, via the user device 102, may view one or more of the orthomosaicimages and define one or more transects corresponding to the area ofland depicted in the one or more orthomosaic images (e.g., in thehabitat or site). The user can identify a number of transects thatshould be included in the area of land depicted in the one or moreorthomosaic images, the shape of each of these transects, and/or adirection of the transects (e.g., if the habitat is along a coastline,the transect may begin at the coastline and the direction of thetransect may be a certain distance inland from the coastline). The imageprocessor 122 can then, for some or all of the pixels in eachorthomosaic image that falls within a transect, use the mappings toidentify a plant species (or animal species, other organisms, dirt,water, etc.) that maps to the brightness temperature of the respectivepixel (e.g., to determine that the respective pixel corresponds to aplant species rather than other material). The image processor 122 canthen identify the plant (or animal, other organism, dirt, water, etc.)at the respective pixel as a separate, individual plant of theidentified plant species (or animal species, other organisms, dirt,water, etc.). Upon identifying individual plants (or animals, otherorganisms, dirt, water, etc.), the image processor 122 may annotate orlabel the corresponding pixel(s) (e.g., place a pinpoint at thecorresponding pixel) such that the user can identify, within a userinterface, individual plants (or animals, other organisms, dirt, water,etc.) and track the individual plants' growth (or lack of growth) overtime (e.g., by viewing different orthomosaic images). Because portionsof the orthomosaic images correspond to particular geographiccoordinates, each individual plant (or animal, other organism, dirt,water, etc.) can be associated with geographic coordinates, a volume ofthe respective individual plant (or animal, other organism, dirt, water,etc.), a height of the respective individual plant (or animal, otherorganism, dirt, water, etc.), a width of the respective individual plant(or animal, other organism, dirt, water, etc.), and/or the like. Thegeographic coordinates, volume, height, width, and/or the like forindividual plants (or animals, other organisms, dirt, water, etc.) canbe stored in a data store of the plant growth prediction system 120(e.g., the coverage percentage data store 129). The user interfacegenerator 131 can retrieve this information to, for example, generateuser interface data that, when rendered by the user device 102, causesthe user device 102 to display the historical growth of one or moreindividual plants (or animals, other organisms, dirt, water, etc.).

A user, via the user device 102, may have provided the image processor122 with a list of plant species that are of interest and/or a list ofplant species that are not of interest. For example, in the context ofhabitat restoration, the user device 102 may transmit to the imageprocessor 122 a list identifying one or more native plant species, fill(e.g., open space, dirt areas, and/or other areas that were damaged orregraded and need to be filled in), one or more invasive species (e.g.,weeds, non-native animals, and/or other objects blown in fromsurrounding areas), and/or plant diversity (e.g., the number ofdifferent types of plant species that are present in the site) as itemsto monitor. Optionally, the user device 102 transmits to the imageprocessor 122 a list of species to ignore. Thus, the image processor 122can then, for each orthomosaic image, use the identified plant species(or animal species, other organisms, dirt, water, etc.) to determine apercentage of the site that is covered by the native species, apercentage of the site that is covered by fill, a percentage of the sitethat is covered by an invasive species, a count representing the plantdiversity in the site, and/or the like. Because each orthomosaic imageis associated with a flight that occurred at a particular time, theimage processor 122 can associate the determined coverage percentagesand/or plant diversity with the time (e.g., day) that the correspondingflight took place. The image processor 122 can store the determinedcoverage percentages and/or plant diversity in the coverage percentagedata store 129 in an entry associated with the associated time and thesite.

In further embodiments, the image processor 122 can use deduplicationtechniques to reduce the likelihood that a plant is double-counted whendetermining the coverage percentages. For example, the image processor122 can use the geographic coordinates included within the metadataassociated with an image taken during a first flight to establish alocation of a plant. As an illustrative example, each pixel in the imagemay map to a set of geographic coordinates according to the metadata.The image processor 1222 can then map each pixel in the orthomosaicimage to a set of geographic coordinates. The image processor 122 maythen identify a plant species in a manner as described herein, where theimage processor 122 maps a pixel to a plant species. Because the pixelalso maps to a set of geographic coordinates, the image processor 122can map the set of geographic coordinates to the plant species. Whenprocessing a new orthomosaic image generated as a result of anadditional flight, the image processor 122 repeat the same process toidentify a plant species location and determine a geographiccoordinate-to-plant species mapping. Thus, by repeating the process, theimage processor 122 can avoid double-counting a plant when determiningthe coverage percentages.

In further embodiments, the image processor 122 can implement techniquesto remove objects from a generated orthomosaic image. For example, theimage processor 122 may be instructed to ignore certain plant and/oranimal species. Using the mappings stored in the plant profile datastore 127, the image processor 122 can identify pixels that correspondto a plant or animal species to ignore and, for example, remove anylabels, shading, or annotations that indicates a brightness temperatureof those pixels.

While the aerial vehicle controller 121 and the image processor 122 aredepicted as being located internal to the plant growth prediction system120, this is not meant to be limiting. For example, the aerial vehiclecontroller 121 and/or the image processor 122 (or a component thatimplements at least a portion of the image processor 122 functionalitydescribed herein) may be internal to another computing device present atthe site where the aerial vehicle 130 is located. Alternatively, theplant growth prediction system 120 (or one or more components of theplant growth prediction system 120) may itself be present at the sitewhere the aerial vehicle 130 is located.

The diagnostic model generator 123 may then retrieve the native speciescoverage percentages of a site stored in the coverage percentage datastore 129. For example, the diagnostic model generator 123 can retrievethe native species coverage percentages once the image processor 122 hasdetermined the values, when a user device 102 transmits a request for astatus report of the site (e.g., current plant growth levels and/orpredicted future plant growth levels), when the plant growth predictionsystem 120 has a low CPU usage (e.g., less than 50%), etc. Using theretrieved native species coverage percentages, the diagnostic modelgenerator 123 can generate a diagnostic model. The diagnostic model canbe used to predict future native species growth (e.g., represented by acoverage percentage). For example, the diagnostic model generator 123can perform a linear regression analysis of the native species coveragepercentages, a cubic polynomial regression analysis of the nativespecies coverage percentages, and/or the like to generate the diagnosticmodel.

As an illustrative example, the diagnostic model generator 123 maycalculate an average of the native species coverage percentages and anaverage of the times associated with the native species coveragepercentages. The diagnostic model generator 123 may define the linearregression as follows:

A=y−Bx   (1)

where y represents an average percentage coverage, x represents anaverage time, and B is defined as follows:

$\begin{matrix}{B = \frac{S_{xy}}{S_{xx}}} & (2)\end{matrix}$

where S_(xx) is defined as follows:

$\begin{matrix}{S_{xx} = {\frac{\sum x_{t}^{2}}{n} - {\overset{\_}{x}}^{2}}} & (3)\end{matrix}$

and where S_(xy) is defined as follows:

$\begin{matrix}{S_{xy} = {\frac{\sum{x_{t}y_{t}}}{n} - \overset{\_}{xy}}} & (4)\end{matrix}$

The diagnostic model generator 123 can apply Equations (1) through (4)using the calculated average native species coverage percentages and thecalculated average times to generate a linear regression diagnosticmodel.

The diagnostic model generator 123 can further generate a diagnosticmodel for fill, an invasive species, and/or plant diversity using thesame techniques. The diagnostic model generator 123 can then store thediagnostic models in the model data store 128 in an entry associatedwith the site.

The diagnostic model(s) may each output a coverage percentage or plantdiversity count as a function of time. Thus, the plant growth predictor124 may then retrieve one or more diagnostic models corresponding to thesite from the model data store 128 and use the diagnostic model(s) topredict future coverage percentages and/or a plant diversity count atvarious times in the future. For example, the plant growth predictor 124can use the diagnostic model corresponding to a native species topredict a time at which the native species will have a coveragepercentage corresponding to a desired coverage percentage (e.g., acoverage percentage that indicates that the habitat restoration iscomplete). As an illustrative example, the plant growth predictor 124can identify a time value that corresponds with a value of the desiredcoverage percentage that falls along a trend line of the native speciesdiagnostic model. Alternatively or in addition, the plant growthpredictor 124 can use the native species diagnostic model to determine apredicted coverage percentage at set times in the future (e.g., 1 yearfrom a current date, 5 years from a current date, etc.).

The plant growth predictor 124 can package the coverage percentageand/or plant diversity predictions (along with historical and/or currentcoverage percentages and/or plant diversity numbers) into a report andtransmit the report to a user device 102 (e.g., either the same userdevice 102 that provided the flight path parameters or a different userdevice 102). The plant growth predictor 124 can transmit the report atthe request of the user device 102.

In addition, the plant growth predictor 124 can modify one or moreorthomosaic images to indicate certain information. For example, theplant growth indicator 124 can annotate a most-recent orthomosaic imageto indicate areas where a native species is growing and areas in whichthe native species needs to grow (e.g., in fill areas) to meet a desiredcoverage percentage. As an illustrative example, the plant growthindicator 124 can retrieve current coverage percentages from thecoverage percentage data store 129. The plant growth indicator 124 canthen determine a difference between the current native species coveragepercentage and a desired native species coverage percentage to identifya diagnostic variance (e.g., represented as a percentage). Thus, thediagnostic variance may represent a percentage of the site that needs tobe filled with the native species to meet the desired native speciescoverage percentage. The plant growth indicator 124 can annotate aportion of the orthomosaic image corresponding to fill areas and/orinvasive species that is a percentage of the site equal to thediagnostic variance, thereby indicating that the annotated portion needsto be filled with the native species instead to meet the desired nativespecies coverage percentage. The plant growth indicator 124 can alsotransmit the modified orthomosaic image(s) to the user device 102.

The user interface generator 131 can generate user interface data andtransmit the user interface data to a user device 102. The userinterface data, when rendered by the user device 102, may cause the userdevice to display a site, navigation controls for controlling the flightof the aerial vehicle 130, and/or other selectable flight pathparameters. Example user interfaces that may be displayed by the userdevice 102 based on the user interface data generated by the userinterface generator 131 are depicted in FIGS. 5A-5B.

In further embodiments, the plant growth prediction system 120 may beinstructed to determine current levels and/or predict future levels fortwo or more different types of objects. For example, the plant growthprediction system 120 may be instructed to determine current and/orpredict future growth of a first plant species and may be instructed todetermine current and/or predict future counts of a first bird species.It may be that the ideal altitude to measure such information may bedifferent for each type of object. Thus, the user device 102 may provideflight path parameters that indicate two different altitudes for thesame flight path. Thus, upon receiving instructions from the aerialvehicle controller 121, the aerial vehicle 130 may fly along the flightpath at the lower altitude in a first pass and then fly along the flightpath at the higher altitude in a second pass, or vice-versa. While thefirst pass may be for measuring levels for a first object, the imageprocessor 122, the diagnostic model generator 123, and/or the plantgrowth predictor 124 may nonetheless use images captured at eachaltitude in order to form the orthomosaic image, generate the diagnosticmodel(s) and/or generate predictions, respectively.

In further embodiments, the plant growth prediction system 120 cancontrol operation of the water system 160. For example, the water system160 may be the water system for a municipality, state, or othergeographic region. The water system 160 may include one or more pipes162 that are controlled by one or more valves 164. The image processor122 can process an orthomosaic images and/or a thermographic image toidentify underground and/or above ground leaks originating from one ormore of the pipes 162. As described herein, the thermographic imagecapture light, such as infrared light, invisible to humans. Thus, thethermographic image (and therefore the orthomosaic image) may depictobjects present below the surface or other objects that are otherwiseinvisible to humans. In one example, such objects can include the flowof water below a sidewalk, below the pavement, and/or the like, thatresult from a leak or water main break. The image processor 122 mayidentify the flow of water based on comparing the shape and/or color ofthe pixels present in the thermographic or orthomosaic image with knownshapes and/or colors of water (e.g., the shape may be a thin andsnake-like, similar to a stream or river, and the color may be withinthe blue color spectrum or any other color representing the wavelengthof light emitted by water). The image processor 122 may furtherrecognize one or more pipes 162 in the thermographic or orthomosaicimage (e.g., based on comparing objects in the image to known shapesand/or colors of pipes 162), thereby allowing the image processor 122 toidentify the specific pipe 162 from which the water is flowing. Once theimage processor 122 identifies the pipe 162 from which the water isflowing, the image processor 122 can generate and transmit a message toa valve 164 corresponding to the identified pipe 162 via the network110, where receipt of the message causes the valve 164 to shut off theflow of water through the pipe 162. Accordingly, the plant growthprediction system 120 can automatically detect a leak and transmitinstructions to cause the flow of water to stop such that the leak canbe fixed.

The flight path data store 125 stores flight path parameters for varioussites. While the flight path data store 125 is depicted as being locatedinternal to the plant growth prediction system 120, this is not meant tobe limiting. For example, not shown, the flight path data store 125 canbe located external to the plant growth prediction system 120.

The image data store 126 stores images captured by the aerial vehicle130 and/or orthomosaic images generated by the image processor 122. Theimages may be stored in entries associated with a site and a time and/orflight identification identifying when the images were captured. Whilethe image data store 126 is depicted as being located internal to theplant growth prediction system 120, this is not meant to be limiting.For example, not shown, the image data store 126 can be located externalto the plant growth prediction system 120.

The plant profile data store 127 stores mappings of plant species tobrightness temperatures. The plant profile data store 127 may also storemappings of other organic and inorganic matter, such as animals,organisms other than plants and animals, dirt, water, etc., tobrightness temperatures. The plant profile data store 127 may furtherstore emissivity levels for various plant species, animal species,organisms other than plants and animals, dirt, water, etc. such that theemissivity levels could be used by the image processor 122 to adjust theemissivity sensitivity levels of captured thermographic images. Whilethe plant profile data store 127 is depicted as being located internalto the plant growth prediction system 120, this is not meant to belimiting. For example, not shown, the plant profile data store 127 canbe located external to the plant growth prediction system 120.

The model data store 128 stores diagnostic models generated by thediagnostic model generator 123. Each diagnostic model may be stored inan entry associated with a site. While the model data store 128 isdepicted as being located internal to the plant growth prediction system120, this is not meant to be limiting. For example, not shown, the modeldata store 128 can be located external to the plant growth predictionsystem 120.

The coverage percentage data store 129 stores coverage percentagesand/or plant diversity determined by the image processor 122. Thecoverage percentages may be stored in entries associated with a site anda time. Optionally, the coverage percentage data store 129 may alsostore plant material data, such as the geographic coordinates, volume,height, width, and/or the like of individual plants. The plant materialdata may be stored in entries associate with a site and/or time. Whilethe coverage percentage data store 129 is depicted as being locatedinternal to the plant growth prediction system 120, this is not meant tobe limiting. For example, not shown, the coverage percentage data store129 can be located external to the plant growth prediction system 120.

Example Plant Health Determination Use Case

As described herein, the plant growth prediction system 120 may beconfigured to determine current and/or predict future plant healthinstead of or in addition to predicting future plant growth. Forexample, the aerial vehicle controller 121 may receive flight pathparameters from the user device 102 via the network 110. Once the aerialvehicle controller 121 determines that the aerial vehicle 130 shouldconduct a flight at a current time, a project member may bring theaerial vehicle 130 to the site (e.g., based on a reminder provided bythe plant growth prediction system 120). The aerial vehicle controller121 can transmit the flight path parameters to the aerial vehicle 130over the network 110 and instruct the aerial vehicle 130 (e.g., theflight path controller 138) to begin the flight. The camera(s) 132and/or a network interface (not shown) of the aerial vehicle 130 maytransmit captured images to the image processor 122 in real-time (e.g.,as the images are captured) and/or after the flight is complete.

The image processor 122 may implement a process to convert the capturedimages into plant health images. For example, the image processor 122can process a captured image pixel by pixel. Each pixel may have an RGBcolor represented by a red value, a green value, and a blue value. Foreach pixel, the image processor 122 can identify the green value of theRGB color. The green value of the RGB color may indicate a relativehealth of a plant and/or whether the pixel depicts a portion of a plantor another object. Based on the magnitude of the green value, the imageprocessor 122 can assign another RGB color to the pixel and convert thepixel from the original RGB color to the newly assigned RGB color. In anembodiment, the assigned RGB color may be a color within a colorspectrum between red (e.g., RGB hexadecimal color #FF0000) and green(e.g., RGB hexadecimal color #00FF00), where orange (e.g., RGBhexadecimal color #FFA500) represents a middle color between red andgreen in the color spectrum. The higher the original green value of thepixel, the closer the newly assigned RGB color will be to green (e.g.,RGB hexadecimal color #00FF00) and the farther the newly assigned RGBcolor will be from red (e.g., RGB hexadecimal color #FF0000). As anillustrative embodiment, if the original RGB color of the pixel is RGBhexadecimal color #340067, then the newly assigned RGB color may be RGBhexadecimal color #FF0000. If the original RGB color of the pixel is RGBhexadecimal color #1EFFB0, then the newly assigned RGB color may be RGBhexadecimal color #00FF00. The image processor 122 can use other similartechniques, such as visible atmospherically resistant index (VARI) ornormalized difference vegetation index (NDVI), to convert the pixels ofthe captured images. Thus, the image processor 122 may normalize the RGBcolor of each pixel of a captured image to an RGB color within the colorspectrum described above. By normalizing the RGB color of each pixel ofa captured image, the image processor 122 may produce a converted imagethat indicates plant health and/or a chemical composition of variousplants (e.g., chlorophyll levels of a plant, nitrogen levels of a plant,etc.).

Alternatively, the image processor 122 can transmit the captured imagesto an external system (not shown) and the external system can processthe captured images to convert the pixels using the same or similarprocess. The external system can then transmit the converted images tothe image processor 122.

The captured images that have pixels converted by the image processor122 and/or external system may be referred to herein as plant healthimages. FIG. 6A depicts an example of a plant health image (shown asplant health data 620). The image processor 122 may further process theplant health images. For example, the image processor 122 can identifygeographic coordinates corresponding to portions of the plant healthimage in which it appears that plants are unhealthy. An unhealthy plantmay be any plant corresponding to one or more pixels that have a valueless than a threshold value (e.g., less than RGB hexadecimal color#ADFF2F, less than RGB hexadecimal color #FFA500, etc.) within theabove-described color spectrum. In some embodiments, sprinkler heads 156of the irrigation system 150 or the irrigation system 150 itself may belocated at specific geographic coordinates. Thus, the image processor122 can transmit a message to a controller 152 of the irrigation system150 (or an external network-based system, not shown, that manages theirrigation system 150) via the network 110 indicating the geographiccoordinates corresponding to unhealthy plants. The controller 152 maymanage the activation and deactivation of valves 154 controlling theflow of water to the sprinkler heads 156 and/or may control theactivation of the sprinkler heads 156 themselves using a wateringschedule. Thus, receipt of the message (either from the image processor122 or the external network-based system) may cause the controller 152of the irrigation system 150 to adjust its watering schedule such thatthe sprinkler heads 156 corresponding to the received geographiccoordinates and/or the valves 154 controlling the flow of water to thesprinkler heads 156 corresponding to the received geographic coordinatesare activated more frequently, may cause the controller 152 of theirrigation system 150 to automatically activate at least the sprinklerheads 156 corresponding to the received geographic coordinates and/orthe valves 154 controlling the flow of water to the sprinkler heads 156corresponding to the received geographic coordinates such that thesprinkler heads 156 spray water, and/or the like. Accordingly, theprocessing of the plant health images performed by the image processor122 may result in an irrigation system 150 watering unhealthy plantsmore often.

Alternatively or in addition, the image processor 122 may process theplant health images to identify the height, width, volume, area, and/orcanopy percentages of plants. For example, the image processor 122 canuse object recognition techniques to identify individual plants (e.g.,the image processor 122 can identify individual plants by identifyingpixels that have a similar color, such as colors that are within athreshold value of each other). Once a plant is identified, the imageprocessor 122 determines a width, volume, and/or area of the plant(e.g., based on the scale of the plant health image). For example, theimage processor 122 can identify a boundary of the plant based on thepixel colors (e.g., a difference in pixel colors above a threshold valueindicates a boundary of the plant) to determine the width, volume,and/or area of the plant. The image processor 122 can further determinea canopy percentage of the plant by measuring the area of the plant as apercentage of the total area of the site.

In addition, as the aerial vehicle 130 captures images, the aerialvehicle 130 may track an altitude of the aerial vehicle 130 (e.g.,relative to the ground) and use a RADAR detector or other similar deviceto identify a distance between the aerial vehicle 130 and an object(e.g., a plant, the ground, water, etc.) below the aerial vehicle 130.At each location, the aerial vehicle 130 can subtract the identifieddistance from the tracked altitude to identify a height of an objectbelow the aerial vehicle 130. Alternatively, the aerial vehicle 130 cantransmit the tracked altitude and the identified distance to the imageprocessor 122 and the image processor 122 can subtract the identifieddistance from the tracked altitude to identify a height of an objectbelow the aerial vehicle 130 at various locations. In this way, theaerial vehicle 130 and/or image processor 122 can determine a height orheights (e.g., branches and leaves may be at different heights) ofplants.

The image processor 122 can compare the determined height, width,volume, area, and/or canopy percentage of a plant or a group of plantswithin a geographic area to threshold heights, widths, volumes, areas,and/or canopy percentages to adjust lighting and/or lighting schedules,to adjust watering and/or watering schedules, and/or to identify whenplants need to be pruned. For example, the image processor 122 cancompare the height, width, volume, area, and/or canopy percentage of oneplant (or one group of plants) against a threshold height, width,volume, area, and/or canopy percentage. If one or more of the height,width, volume, area, and/or canopy percentage values is greater than oneor more of the threshold height, width, volume, area, and/or canopypercentage values by a threshold value or percentage, then this mayindicate that the area beneath this plant (or group of plants) isgenerally dark. Thus, the image processor 122 can transmit a message toa lighting system (or an external network-based system, not shown, thatmanages the lighting system) via the network 110 indicating thegeographic coordinates of this plant (or group of plants). Receipt ofthe message (either from the image processor 122 or the externalnetwork-based system) may cause the lighting system to adjust itsschedule such that lights corresponding to the received geographiccoordinates are activated earlier in the day and/or for a longer periodof time, may cause the lighting system to automatically activate atleast the lights corresponding to the received geographic coordinates,and/or the like. The image processor 122 may also generate anotification for transmission to a user device 102 via the network 110(e.g., a push notification) indicating that the plant (or group ofplants) need to be pruned.

Conversely, if one or more of the height, width, volume, area, and/orcanopy percentage values is less than one or more of the thresholdheight, width, volume, area, and/or canopy percentage values by athreshold value or percentage, then this may indicate that the areabeneath this plant (or group of plants) is generally light and the imageprocessor 122 can transmit no message or a message to the lightingsystem (or the external network-based system) to perform the oppositeoperation (e.g., turn on the lights later and/or for a shorter period oftime, automatically turn off the lights, etc.). Thus, the processingperformed by the image processor 122 can be used to conserve energy viathe efficient use of lighting. The image processor 122 may also generatea notification for transmission to a user device 102 via the network 110(e.g., a push notification) indicating that the plant (or group ofplants) do not need to be pruned, the plant (or group of plants) shouldbe pruned later than scheduled, and/or additional plants should beplanted in the corresponding geographic area. In addition, as describedabove, the image processor 122 can generate and transmit a message tocause the irrigation system 150 to water the plant (or group of plants)automatically and/or more frequently.

In some cases, the user interface generator 131 can generate userinterface data that, when rendered by the user device 102, causes theuser device 102 to display a user interface providing information onplant height, width, volume, area, and/or canopy percentage and/or toolsfor allowing a user to manually measure such information. For example,the user interface generator 131 may receive the captured images and/orthe plant health images from the image processor 122 for display.Example user interfaces are depicted in FIGS. 8A-8B and are described ingreater detail below.

Furthermore, the plant growth prediction system 120 can use similartechniques as described above with respect to predicting plant growth topredict plant health. For example, the diagnostic model generator 123may receive, for a specific plant, the health of the plant (e.g., asrepresented by RGB hexadecimal colors) determined over a period of timevia various images captured over a period of time. Using the planthealth information, the diagnostic model generator 123 can generate adiagnostic model. The diagnostic model can be used to predict futureplant health for that plant (e.g., represented by an RGB hexadecimalcolor). For example, the diagnostic model generator 123 can perform alinear regression analysis of the plant health, a cubic polynomialregression analysis of the plant health, and/or the like to generate thediagnostic model. The diagnostic model generator 123 may generate adifferent diagnostic model for each plant or for different sets ofplants (e.g., plants that are within proximity of each other). Thediagnostic model generator 123 can store the diagnostic models in themodel data store 128.

The diagnostic models may output a plant health value as a function oftime. Thus, the plant growth predictor 124 may then retrieve adiagnostic model for each plant at a site from the model data store 128and use the diagnostic models to predict future plant health for one ormore plants at various times in the future. The plant growth predictor124 can package the predicted future plant health values into a reportand/or provide the predicted future plant health values to the userinterface generator 131 such that the user interface generator 131 cangenerate user interface data that, when rendered by the user device 102,causes the user device 102 to display the predicted future plant healthvalues. As described above, the plant growth predictor 124 can alsomodify one or more captured images to indicate the predicted futureplant health values. The plant growth indicator 124 can transmit themodified captured image(s) to the user device 102.

As described above, the image processor 122 can stitch the imagesreceived from the aerial vehicle 130 together to form a single stitchedimage. The image processor 122 can stitch the images before or after theimages are converted into plant health images.

Example Block Diagrams for Determining and Predicting Plant Growth

FIG. 2 is a flow diagram illustrating the operations performed by thecomponents of the operating environment 100 of FIG. 1 to generate anorthomosaic image for an initial flight, according to one embodiment. Asillustrated in FIG. 2, the user device 102, based on input from a user,sets flight path parameters and transmits the flight path parameters tothe aerial vehicle controller 121 at (1). The aerial vehicle controller121 may then store the flight path parameters in the flight path datastore 125 at (2). Before, during, or after storing the flight pathparameters, the aerial vehicle controller 121 can instruct the aerialvehicle 130 at (3) to capture images using the flight path parameters.

In response to receiving the instruction to capture images, the aerialvehicle 130 can begin a flight and capture images at (4). As images arecaptured and/or after the flight is complete, the aerial vehicle 130 cantransmit the captured images to the image processor 122 at (5).

The image processor 122 can generate an orthomosaic image using theretrieved images at (6). For example, the retrieved images may be boththermographic images and high-resolution images. The image processor 122can stitch the thermographic images together and can stitch thehigh-resolution images together. The image processor 122 can thencombine the stitched images to form an orthomosaic image in which theimage is geometrically corrected with a uniform scale. The imageprocessor 122 can then store the orthomosaic image in the image datastore 126 at (7).

FIGS. 3A-3B are flow diagrams illustrating the operations performed bythe components of the operating environment 100 of FIG. 1 to predictplant growth after a flight that follows the initial flight, accordingto one embodiment. As illustrated in FIG. 3A, the aerial vehiclecontroller 121 can retrieve flight path parameters from the flight pathdata store 125 at (1). For example, because this is a flight after theinitial flight, the user device 102 may have already provided flight setparameters. The aerial vehicle controller 121 can then query the flightpath data store 125 using an identification of the site to retrieve theappropriate flight path parameters. The aerial vehicle controller 121may retrieve the flight path parameters at a time previously indicatedby the user device 102 (e.g., the user device 102 may provide a set oftimes for using the aerial vehicle 130 to capture images and/or anindication when an event corresponding to a flight has commenced, suchas when a site has been impacted). The aerial vehicle controller 121 maythen instruct the aerial vehicle 130 at (2) to capture images using theflight path parameters.

In response to receiving the instruction to capture images, the aerialvehicle 130 can begin a flight and capture images at (3). As images arecaptured and/or after the flight is complete, the aerial vehicle 130 cantransmit the captured images to the image processor 122 at (4).

The image processor 122 can generate an orthomosaic image using theretrieved images at (5). For example, the retrieved images may be boththermographic images and high-resolution images. The image processor 122can stitch the thermographic images together and can stitch thehigh-resolution images together. The image processor 122 can thencombine the stitched images to form an orthomosaic image in which theimage is geometrically corrected with a uniform scale.

The image processor 122 can then retrieve previous orthomosaic imagesfrom the image data store 126 at (6). In each orthomosaic image, theimage processor 122 can identify at (7) the brightness temperaturelevels of each pixel.

Before, during, or after identifying the brightness temperature levels,the image processor 122 can retrieve plant profile data from the plantprofile data store 127 at (8). For example, the plant profile data mayinclude mappings of plant species to brightness temperatures. Using theplant profile data, the image processor 122 can identify plant speciescorresponding to each pixel in each orthomosaic image and, therefore,the plant species that are present at the site at (9).

At a previous time, the user device 102 may provide the image processor122 with a list of specific plant species to examine, such as anidentification of native species, fill, and/or invasive species. Theimage processor 122 can use this information along with the identifiedplant species to determine at (10), in each orthomosaic image, a nativespecies coverage percentage, a fill coverage percentage, and/or aninvasive species coverage percentage. In addition, the image processor122 can use the identified plant species information to determine, ineach orthomosaic image, plant diversity at the site.

As illustrated in FIG. 3B, the image processor 122 stores the coveragepercentages in the coverage percentage data store 129 at (11). At alater time, such as when the user device 102 requests a report, thediagnostic model generator 123 can retrieve the coverage percentages at(12). The diagnostic model generator 123 can use the coveragepercentages to generate one or more diagnostic models at (13). Forexample, the diagnostic model generator 123 may generate a nativespecies diagnostic model, a fill diagnostic model, an invasive speciesdiagnostic model, and/or a plant diversity diagnostic model. Thediagnostic model generator 123 can store the generated diagnosticmodel(s) in the model data store 128 at (14).

The plant growth predictor 124 can retrieve the generated diagnosticmodel(s) at (15) and use the diagnostic model(s) to predict future plantgrowth at the site at (16). For example, the plant growth predictor 124can predict a time when a desired native species coverage percentagewill be achieved. The plant growth predictor 124 can retrieve coveragepercentages from the coverage percentage data store 129 at (17) andgenerate a report depicting the current and predicted future plantgrowth at (18) (e.g., where the current and predicted future plantgrowth is represented as a coverage percentage).

In further embodiments, the plant growth prediction system 120 caninstruct other devices to perform actions in response to the results ofthe report. As an illustrative example, if the report generated for acurrent time period indicates that the current plant growth is 0%,whereas a previously generated report predicted that the plant growthduring the current time period would be 10%, then the plant growthprediction system 120 can instruct a sprinkler system (e.g., theirrigation system 150) to modify a watering schedule such that thesprinklers (e.g., the sprinkler heads 156) in the sprinkler system waterthe site more often, instruct a vehicle or other like apparatus torelease additional fertilizer in the site, transmit a notification toanother device (e.g., the user device 102) instructing a user to addmore water and/or fertilizer to the site or to otherwise adjust a plantgrowth plan, and/or the like.

In addition, the plant growth predictor 124 can retrieve an orthomosaicimage from the image data store 126 at (19). For example, the plantgrowth predictor 124 may retrieve the latest orthomosaic image. Theplant growth predictor 124 can then modify the orthomosaic image at (20)to indicate current and predicted future plant growth. For example, theorthomosaic image can be shaded with colors to differentiate betweencurrent and predicted future plant growth. The plant growth predictor124 may then transmit the report and/or modified orthomosaic image tothe user device 102 at (21).

Example Plant Growth Prediction Routine

FIG. 4 is a flow diagram depicting a plant growth prediction routine 400illustratively implemented by a plant growth prediction system,according to one embodiment. As an example, the plant growth predictionsystem 120 of FIG. 1 can be configured to execute the plant growthprediction routine 400. The plant growth prediction routine 400 beginsat block 402.

At block 404, flight path parameters are received. For example, theflight path parameters can include a flight path, a shooting angle, acapture mode, a gimbal pitch angle, an end-mission action, and/or thelike. The flight path parameters can be received from a user device 102or the flight path data store 125.

At block 406, an aerial vehicle is instructed to captures imagesaccording to the flight path parameters. For example, the aerial vehiclemay capture images using a thermal camera and a high-resolution camera.

At block 408, images are received from the aerial vehicle. For example,the images may be received in real-time and/or after the flight iscomplete.

At block 410, an orthomosaic image is generated using the receivedimages. For example, the images captured by the thermal camera may becombined and the images captured by the high-resolution camera may becombined. The combined images may then be merged to form the orthomosaicimage.

At block 412, the orthomosaic image is processed to identify a coveragepercentage for a first plant species. For example the first plantspecies may be a native species of the site.

At block 414, a diagnostic model is generated using the identifiedcoverage percentage. For example, the identified coverage percentage andone or more historical coverage percentages may be used to generate thediagnostic model.

At block 416, future plant growth is predicted using the diagnosticmodel. For example, a time when the plant growth reaches a desiredcoverage percentage may be predicted. After the future plant growth ispredicted, the plant growth prediction routine 400 is complete, as shownat block 418.

Example User Interfaces

FIG. 5A illustrates a user interface 500 displaying a site 520 and alist 550 of basic flight path parameters. The user interface 500 may bedisplayed by a user device 102 based on a rendering of user interfacedata generated and provided by the plant growth prediction system 120(e.g., the user interface generator 131).

As illustrated in FIG. 5A, the user interface 500 displays a window 510that includes an image depicting the site 520. The depiction of the site520 is modified with a flight path 522 of the aerial vehicle 130 thatoverlays the image of the site 520. As described herein, a user cangenerate the flight path 522 by, for example, dragging a cursor or touchinput across the user interface 500. Alternatively or in addition, theuser can enter in the user interface 500 a set of geographic coordinatesand an order in which the geographic coordinates are to be reached,thereby forming the flight path 522. Each time a direction of the flightpath 522 changes, a waypoint (e.g., represented by white circles) may bedepicted in the flight path 522 at the point of the direction change. Inaddition, a starting position 524 of the flight path 522 may beindicated in the window 510 as well as a current location 530 of theaerial vehicle 130 if the aerial vehicle 130 is already in flight.

The window 510 may further include a box 540 indicating currentparameters of the aerial vehicle 130. For example, the box 540 mayinclude the direction, speed, latitude, longitude, and/or altitude ofthe aerial vehicle 130.

The list 550 of basic flight path parameters may include a mission type(e.g., 3DMap, 2DMap, etc.), an estimated flight time, a flight length(e.g., a length of the flight path 522), a number of waypoints in theflight path 522, a camera model, a shooting angle, a capture mode, aflight course mode, a aerial vehicle 130 speed, an aerial vehicle 130altitude, geographic coordinates of a starting point (e.g., latitude andlongitude), and/or a resolution of the camera.

There may be several types of shooting angles. For example, the parallelto main path shooting angle may cause a camera 132 to be positioned suchthat a lens of the camera 132 faces directly down (e.g., 90 degreesstraight down) and is parallel with the ground and the vertical to mainpath shooting angle may cause a camera 132 to be positioned such that alens of the camera 132 faces directly ahead or to the side of the aerialvehicle 130 and is perpendicular with the ground. In addition, theshooting angle may be selected to be an angle between parallel to mainpath and vertical to main path.

There may also be several types of capture modes. For example, the hoverand capture at point capture mode results in a camera 132 capturing animage at each waypoint, the capture at equal time intervals capture moderesults in a camera 132 capturing an image in set time intervals, andthe capture at equal distance intervals results in a camera 132capturing an image every threshold distance.

FIG. 5B illustrates a user interface 560 displaying the site 520 and alist 570 of advanced flight path parameters. The user interface 560 maybe displayed by a user device 102 based on a rendering of user interfacedata generated and provided by the plant growth prediction system 120(e.g., the user interface generator 131).

As illustrated in FIG. 5B, the list 570 of advanced flight pathparameters can include some basic flight parameters (e.g., mission type,an estimated flight time, a flight length, a number of waypoints in theflight path 522, and geographic coordinates of a starting point), frontoverlap ratio, side overlap ratio, course angle, margin, gimbal pitchangle, and end-mission action.

If the user updates any of the basic or advanced flight path parametersin the user interfaces 500 or 560, this may cause the user device 102 tonotify the aerial flight controller 121 of the update. The aerial flightcontroller 121 may then transmit an instruction to the flight pathcontroller 138 to update the flight path according to the updated flightpath parameter(s). Thus, the user may be able to update the flight pathof the aerial vehicle 130 in real-time as the aerial vehicle 130 is inflight.

In further embodiments, not shown, the user interfaces 500 and/or 560can display images captured by the aerial vehicle 130 as those imagesare captured. The images may be received by the user device 102 from theimage processor 122.

FIG. 6A illustrates a user interface 600 displaying plant health data620 of a site overlaid over a high-resolution image of the site depictedin a window 610. Thus, the combined images depicted in the window 610may be an orthomosaic image. The user interface 600 may be displayed bya user device 102 based on a rendering of user interface data generatedand provided by the plant growth prediction system 120 (e.g., the plantgrowth predictor 124). As an example, the plant growth prediction system120 may be configured to determine current and/or predict future planthealth for the purposes of the embodiment disclosed in FIG. 6A. Theplant growth prediction system 120 may be configured to determinecurrent and/or predict future plant health instead of or in addition topredicting future plant growth. The depicted plant health data 620 maybe current plant health determined by the plant growth prediction system120 and/or future plant health predicted by the plant growth predictionsystem 120.

As described herein, the plant growth predictor 124 can transmit areport and/or a modified orthomosaic image to the user device 102 toshow current levels and/or predicted future levels. In furtherembodiments, the modified orthomosaic image can be appended withadditional information that can be displayed in a user interface, suchas the user interface 600. For example, as illustrated in FIG. 6A, themodified orthomosaic image can be appended with information identifyinga size of the site (e.g., 0.643 acres), a date, a histogram 630, andoptions to modify the format in which the modified orthomosaic image isdisplayed (e.g., color options can be changed to jet, black and white,grayscale, etc.; intensity options can be changed to low, medium, high,etc.). The histogram 630 may show a quantity or percentage of plants inthe site that have a particular health. Each health level may correspondto a shaded color (e.g., an RGB value, a grayscale value, etc.). Thehealth levels can be absolute values or normalized (e.g., on a scalefrom −1 to 1, where −1 is the unhealthiest level and 1 is the healthiestlevel).

The orthomosaic image may be modified with box 625. Box 625 may be addedto the orthomosaic image by the plant growth predictor 124 to indicatethat if the plants in the portion of the site within box 625 becomehealthier (e.g., to a certain plant health level), then the desiredcoverage percentage will be reached.

FIG. 6B illustrates a user interface 650 displaying elevation 660 of asite overlaid over a high-resolution image of the site depicted in thewindow 610. Thus, the combined images depicted in the window 610 mayalso be an orthomosaic image. The user interface 650 may be displayed bya user device 102 based on a rendering of user interface data generatedand provided by the plant growth prediction system 120 (e.g., the plantgrowth predictor 124). As an example, the plant growth prediction system120 may be configured to determine current and/or predict future soilelevations (e.g., in light of possible erosion) for the purposes of theembodiment disclosed in FIG. 6B. The depicted elevation 620 may becurrent elevation determined by the plant growth prediction system 120and/or future elevation predicted by the plant growth prediction system120.

As illustrated in FIG. 6B, the orthomosaic image is appended with a sizeof the site (e.g., 0.643 acres), a date, a histogram 670, and options toannotate the orthomosaic image. The histogram 670 may show a quantity orpercentage of terrain in the site that has a particular elevation. Eachelevation level may correspond to a shaded color (e.g., an RGB value, agrayscale value, etc.).

FIG. 7 illustrates a user interface 700 displaying individuallyidentified plants in a site overlaid over a high-resolution image of thesite depicted in a window 710. The user interface 700 may be displayedby a user device 102 based on a rendering of user interface datagenerated and provided by the plant growth prediction system 120 (e.g.,the user interface generator 131).

As illustrated in FIG. 7, the window 710 includes a transect 720 thatmay have been selected by a user. Within the transect 720, twoindividual plants are labeled: plant 722 and plant 724. The plants 722and 724 may be labeled by the image processor 122 after the imageprocessor 122 compares individual pixels of the high-resolution image tomappings between brightness temperature and plant species. The labeledplants 722 and 724 may be selectable.

In window 730, the user interface 700 may display information for aselected plant. For example, the window 730 may display a name of theselected plant, a species of the plant, geographic coordinates of theselected plant, an elevation at which the selected plant is situated, aheight of the selected plant, a width of the selected plant, and/or avolume of the selected plant. The user may have the option, not shown,of viewing high-resolution images of the transect 720 taken at differenttimes to view the change in plants 722 and/or 724 over time.

FIGS. 8A-8B illustrate a user interface 800 displaying tools foranalyzing plants at a site depicted in a window 810. The user interface800 may be displayed by a user device 102 based on a rendering of userinterface data generated and provided by the plant growth predictionsystem 120 (e.g., the user interface generator 131).

As illustrated in FIG. 8A, the user interface 800 may include a window830 that includes a location button 832, a width button 834, an areabutton 836, and a volume button 838. Selection of the location button832 may allow a user to select a plant in the window 810. Selection ofthe width button 834 may allow a user to measure the width of a plant.For example, the user may be able to place a line 822 in the window 810.Creation of the line 822 causes the window 830 to indicate the length ordistance covered by the line 822 (e.g., 13.45 ft). In addition, creationof the line 822 causes the window 830 to display a line graph 840indicating a height of the plant or plants covered by the line 822. Forexample, plants may include branches, leaves, and/or the like locationat various heights. The plants may also not completely cover a certainarea (when looking at the plant from above) due to gaps betweenbranches, leaves, and/or the like. Thus, the height of a plant may vary.The line graph 840 indicates, for each point along the line 822, theheight of the plant or plants at the respective point along the line822.

Selection of the area button 836 may allow a user to measure an area ofa plant. For example, as illustrated in FIG. 8B, the user may be able tocreate a polygon 824 representing the outer boundaries of a plant.Creation of the polygon 824 causes the window 830 to indicate an area,cut, fill, and/or volume covered by the polygon 824 (e.g., 124 ft², 31.9y³, 3.4 y³, and 23.1 y³, respectively).

Example Plant Health Detection Routine

FIG. 9 is a flow diagram depicting a plant health prediction routine 900illustratively implemented by a plant growth prediction system,according to one embodiment. As an example, the plant growth predictionsystem 120 of FIG. 1 can be configured to execute the plant healthprediction routine 900. The plant health prediction routine 900 beginsat block 902.

At block 904, flight path parameters are received. For example, theflight path parameters can include a flight path, a shooting angle, acapture mode, a gimbal pitch angle, an end-mission action, and/or thelike. The flight path parameters can be received from a user device 102or the flight path data store 125.

At block 906, an aerial vehicle is instructed to captures imagesaccording to the flight path parameters. For example, the aerial vehiclemay capture images using a thermal camera and/or a high-resolutioncamera.

At block 908, images are received from the aerial vehicle. For example,the images may be received in real-time and/or after the flight iscomplete.

At block 910, the received images are converted into plant healthimages. For example, the image processor 122 can use the green value ofthe RGB color of a pixel to identify a new RGB color for the pixel andconvert the pixel to the new RGB color.

At block 912, the plant health images are processed. For example, theimage processor 122 can process the plant health images to identify theheight, width, volume, area, and/or canopy percentages of plants. Theimage processor 122 can then compare the identified heights, widths,volumes, areas, and/or canopy percentages with threshold heights,widths, volumes, areas, and/or canopy percentages.

At block 914, a message is transmitted to an external system to cause anaction to be performed. For example, based on the comparison performedby the image processor 122, the image processor 122 may prepare amessage for transmission to an external system. The external system maybe the irrigation system 150, a lighting system, and/or the like.Receipt of the message may cause the external system to perform anaction, such as adjusting lighting and/or lighting schedules, adjustingwatering and/or watering schedules, and/or notifying when plants need tobe pruned. After the message is transmitted to the external system, theplant health detection routine 900 is complete, as shown at block 916.

Example Aerial Vehicle

FIG. 10 illustrates an exemplary aerial vehicle 130. As illustrated inFIG. 10, the aerial vehicle 130 is an unmanned aerial vehicle in which acamera 132 is coupled to a body 1020 of the aerial vehicle 130 via agimbal 1010. The aerial vehicle 130 further includes four rotors1050A-D. For example, the exemplary aerial vehicle 130 illustrated inFIG. 10 may be the INSPIRE 1 PRO drone. While the exemplary aerialvehicle 130 illustrated in FIG. 10 includes four rotors 1050A-D, this isnot mean to be limiting. For example, the aerial vehicle 130 may includeany number of rotors (e.g., six, eight, ten, twelve, etc.).

The gimbal 1010 may allow the camera 132 to rotate 360 degrees within ahorizontal plane (e.g., a plane that extends from a left side of theaerial vehicle 130 to a right side of the aerial vehicle 130, a planethat extends from a back side of the aerial vehicle 130 to a front sideof the aerial vehicle 130, etc.). As an illustrative example, the gimbal1010 may allow the camera 132 to be positioned such that a lens 1032 ofthe camera 132 faces a right-front rotor apparatus 1040.

Similarly, the gimbal 1010 may allow the camera 132 to rotate at least180 degrees within a vertical plane (e.g., a plane that extends from atop side of the aerial vehicle 130 to a bottom side of the aerialvehicle 130). As an illustrative example, the gimbal 1010 may allow thecamera 132 to be positioned such that the lens 1032 faces a surfacedirectly below the body 1020 of the aerial vehicle 130.

Additional Embodiments

Various example user devices 102 are shown in FIG. 1, including adesktop computer, laptop, and a mobile phone, each provided by way ofillustration. In general, the user devices 102 can be any computingdevice such as a desktop, laptop or tablet computer, personal computer,wearable computer, server, personal digital assistant (PDA), hybridPDA/mobile phone, mobile phone, electronic book reader, set-top box,voice command device, camera, digital media player, and the like. A userdevice 102 may execute an application (e.g., a browser, a stand-aloneapplication, etc.) that allows a user to view captured images, setflight path parameters, modify a flight path during flight, and/or viewpredictions and associated annotated orthomosaic images.

The network 110 may include any wired network, wireless network, orcombination thereof. For example, the network 110 may be a personal areanetwork, local area network, wide area network, over-the-air broadcastnetwork (e.g., for radio or television), cable network, satellitenetwork, cellular telephone network, or combination thereof. As afurther example, the network 110 may be a publicly accessible network oflinked networks, possibly operated by various distinct parties, such asthe Internet. In some embodiments, the network 110 may be a private orsemi-private network, such as a corporate or university intranet. Thenetwork 110 may include one or more wireless networks, such as a GlobalSystem for Mobile Communications (GSM) network, a Code Division MultipleAccess (CDMA) network, a Long Term Evolution (LTE) network, or any othertype of wireless network. The network 110 can use protocols andcomponents for communicating via the Internet or any of the otheraforementioned types of networks. For example, the protocols used by thenetwork 110 may include Hypertext Transfer Protocol (HTTP), HTTP Secure(HTTPS), Message Queue Telemetry Transport (MQTT), ConstrainedApplication Protocol (CoAP), and the like. Protocols and components forcommunicating via the Internet or any of the other aforementioned typesof communication networks are well known to those skilled in the artand, thus, are not described in more detail herein.

Terminology

All of the methods and tasks described herein may be performed and fullyautomated by a computer system. The computer system may, in some cases,include multiple distinct computers or computing devices (e.g., physicalservers, workstations, storage arrays, cloud computing resources, etc.)that communicate and interoperate over a network to perform thedescribed functions. Each such computing device typically includes aprocessor (or multiple processors) that executes program instructions ormodules stored in a memory or other non-transitory computer-readablestorage medium or device (e.g., solid state storage devices, diskdrives, etc.). The various functions disclosed herein may be embodied insuch program instructions, or may be implemented in application-specificcircuitry (e.g., ASICs or FPGAs) of the computer system. Where thecomputer system includes multiple computing devices, these devices may,but need not, be co-located. The results of the disclosed methods andtasks may be persistently stored by transforming physical storagedevices, such as solid state memory chips or magnetic disks, into adifferent state. In some embodiments, the computer system may be acloud-based computing system whose processing resources are shared bymultiple distinct business entities or other users.

Depending on the embodiment, certain acts, events, or functions of anyof the processes or algorithms described herein can be performed in adifferent sequence, can be added, merged, or left out altogether (e.g.,not all described operations or events are necessary for the practice ofthe algorithm). Moreover, in certain embodiments, operations or eventscan be performed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors or processor cores or onother parallel architectures, rather than sequentially.

The various illustrative logical blocks, modules, routines, andalgorithm steps described in connection with the embodiments disclosedherein can be implemented as electronic hardware (e.g., ASICs or FPGAdevices), computer software that runs on computer hardware, orcombinations of both. Moreover, the various illustrative logical blocksand modules described in connection with the embodiments disclosedherein can be implemented or performed by a machine, such as a processordevice, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A processor device can be amicroprocessor, but in the alternative, the processor device can be acontroller, microcontroller, or logic circuitry that implements a statemachine, combinations of the same, or the like. A processor device caninclude electrical circuitry configured to process computer-executableinstructions. In another embodiment, a processor device includes an FPGAor other programmable device that performs logic operations withoutprocessing computer-executable instructions. A processor device can alsobe implemented as a combination of computing devices, e.g., acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor device mayalso include primarily analog components. For example, some or all ofthe rendering techniques described herein may be implemented in analogcircuitry or mixed analog and digital circuitry. A computing environmentcan include any type of computer system, including, but not limited to,a computer system based on a microprocessor, a mainframe computer, adigital signal processor, a portable computing device, a devicecontroller, or a computational engine within an appliance, to name afew.

The elements of a method, process, routine, or algorithm described inconnection with the embodiments disclosed herein can be embodieddirectly in hardware, in a software module executed by a processordevice, or in a combination of the two. A software module can reside inRAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory,registers, hard disk, a removable disk, a CD-ROM, or any other form of anon-transitory computer-readable storage medium. An exemplary storagemedium can be coupled to the processor device such that the processordevice can read information from, and write information to, the storagemedium. In the alternative, the storage medium can be integral to theprocessor device. The processor device and the storage medium can residein an ASIC. The ASIC can reside in a user terminal. In the alternative,the processor device and the storage medium can reside as discretecomponents in a user terminal.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements or steps.Thus, such conditional language is not generally intended to imply thatfeatures, elements or steps are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without other input or prompting, whether thesefeatures, elements or steps are included or are to be performed in anyparticular embodiment. The terms “comprising,” “including,” “having,”and the like are synonymous and are used inclusively, in an open-endedfashion, and do not exclude additional elements, features, acts,operations, and so forth. Also, the term “or” is used in its inclusivesense (and not in its exclusive sense) so that when used, for example,to connect a list of elements, the term “or” means one, some, or all ofthe elements in the list.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, or Z). Thus,such disjunctive language is not generally intended to, and should not,imply that certain embodiments require at least one of X, at least oneof Y, and at least one of Z to each be present.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it can beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As can berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers. The scope of certain embodiments disclosed herein is indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

1.-27. (canceled)
 28. A system for predicting plant health, the systemcomprising: an unmanned aerial vehicle, wherein the unmanned aerialvehicle comprises a camera; and a computing system comprising one ormore computing devices, wherein the computing system is configured tocommunicate with the unmanned aerial vehicle and configured withspecific computer-executable instructions to: instruct the unmannedaerial vehicle to capture a first set of images using the camera whileflying along a flight path; obtain the first set of images captured bythe unmanned aerial vehicle; process the first set of images such thatindividual pixels of the first set of images indicate a relative planthealth; generate a diagnostic model using the indicated relative planthealth; and predict future plant health using the diagnostic model. 29.The system of claim 28, wherein the computing system is furtherconfigured with specific computer-executable instructions to identifyindividual plants in the first set of images using object recognitiontechniques.
 30. The system of claim 29, wherein the computing system isfurther configured with specific computer-executable instructions toidentify a boundary of a first plant in the first set of images based ona difference in pixel colors in the first set of images.
 31. The systemof claim 30, wherein the computing system is further configured withspecific computer-executable instructions to determine a canopypercentage of the first plant based on the boundary of the first plantand a total area of a site.
 32. The system of claim 31, wherein thecomputing system is further configured with specific computer-executableinstructions to transmit a message to a lighting system to cause thelighting system to adjust a lighting schedule based on the determinedcanopy percentage.
 33. The system of claim 28, wherein the computingsystem is further configured with specific computer-executableinstructions to: for each image in the first set of images and for eachpixel in the respective image, identify a green color value of therespective pixel; and convert a color of the respective pixel into a newcolor within a color spectrum based on the identified green color value.34. The system of claim 28, wherein the computing system is furtherconfigured with specific computer-executable instructions to generatethe diagnostic model using the relative plant health indicated by theprocessed first set of images and relative plant health indicated byprocessed second set of images, the second set of images captured priorto the first set of images.
 35. The system of claim 28, wherein thecomputing system is further configured with specific computer-executableinstructions to generate user interface data that, when rendered by auser device, causes the user device to display the predicted planthealth.
 36. The system of claim 28, wherein the computing system isfurther configured with specific computer-executable instructions to:modify a first image in the first set of images to indicate thepredicted future plant health; and transmit the modified first image toa user device.
 37. The system of claim 28, wherein the computing systemis further configured with specific computer-executable instructions to:generate the diagnostic model for a first plant depicted in the firstset of images; generate a second diagnostic model for a second plantdepicted in the first set of images; predict future plant health of thefirst plant using the diagnostic model; and predict future plant healthof the second plant using the second diagnostic model.
 38. The system ofclaim 28, wherein the computing system is further configured withspecific computer-executable instructions to: obtain flight pathparameters from a user device over a network; and instruct the unmannedaerial vehicle to capture the first set of images using the camera whileflying along a flight path according to the flight path parameters. 39.The system of claim 38, wherein the flight path parameters comprise atleast one of geographic coordinates, waypoints, flight length, flighttime, speed, altitude, camera shooting angle, camera capture mode, orcamera resolution.
 40. The system of claim 28, wherein the computingsystem is further configured with specific computer-executableinstructions to: identify geographic coordinates of a portion of a firstimage in the first set of images that corresponds to an unhealthy plantusing the indicated relative plant health; and transmit an instructionfor receipt by an irrigation system corresponding to the geographiccoordinates, wherein receipt of the instruction results in a sprinklerhead in the irrigation system being activated.
 41. Acomputer-implemented method of predicting plant health, the methodcomprising: as implemented by one or more computing devices configuredwith specific computer-executable instructions, obtaining a first set ofimages captured by an aerial vehicle over an area that comprises atleast plant material; processing the first set of images such thatindividual pixels of the first set of images indicate plant health;generating a diagnostic model using the indicated plant health; andpredicting future plant health using the diagnostic model.
 42. Thecomputer-implemented method of claim 41, further comprising identifyingindividual plants in the first set of images using object recognitiontechniques.
 43. The computer-implemented method of claim 42, whereinidentifying individual plants in the first set of images furthercomprises identifying a boundary of a first plant in the first set ofimages based on a difference in pixel colors in the first set of images.44. The computer-implemented method of claim 41, wherein processing thefirst set of images further comprises: for each image in the first setof images and for each pixel in the respective image, identifying agreen color value of the respective pixel; and converting a color of therespective pixel into a new color within a color spectrum based on theidentified green color value.
 45. The computer-implemented method ofclaim 41, wherein generating a diagnostic model further comprisesgenerating the diagnostic model using the plant health indicated by theprocessed first set of images and plant health indicated by processedsecond set of images, the second set of images captured prior to thefirst set of images.
 46. The computer-implemented method of claim 41,further comprising: modifying a first image in the first set of imagesto indicate the predicted future plant health; and transmitting themodified first image to a user device.
 47. The computer-implementedmethod of claim 41, wherein generating the diagnostic model furthercomprises: generating the diagnostic model for a first plant depicted inthe first set of images; generating a second diagnostic model for asecond plant depicted in the first set of images; predicting futureplant health of the first plant using the diagnostic model; andpredicting future plant health of the second plant using the seconddiagnostic model.
 48. The computer-implemented method of claim 41,further comprising: identifying geographic coordinates of a portion of afirst image in the first set of images that corresponds to an unhealthyplant using the indicated plant health; and transmitting a messageintended for an irrigation system corresponding to the geographiccoordinates, wherein receipt of the message results in a sprinkler headin the irrigation system being activated more frequently. 49.Non-transitory, computer-readable storage media comprisingcomputer-executable instructions for predicting plant health, whereinthe computer-executable instructions, when executed by a computersystem, cause the computer system to: obtain a first set of imagesdepicting at least plant material; convert the first set of images intoa first set of plant health images that indicate plant health; generatea diagnostic model using the indicated plant health; and predict futureplant health using the diagnostic model.
 50. The non-transitory,computer-readable storage media of claim 49, wherein thecomputer-executable instructions further cause the computer system toidentify individual plants in the first set of images using objectrecognition techniques.
 51. The non-transitory, computer-readablestorage media of claim 50, wherein the computer-executable instructionsfurther cause the computer system to identify a boundary of a firstplant in the first set of images based on a difference in pixel colorsin the first set of images.
 52. The non-transitory, computer-readablestorage media of claim 49, wherein the computer-executable instructionsfurther cause the computer system to: for each image in the first set ofimages and for each pixel in the respective image, identify a greencolor value of the respective pixel; and convert a color of therespective pixel into a new color within a color spectrum based on theidentified green color value.
 53. The non-transitory, computer-readablestorage media of claim 49, wherein the computer-executable instructionsfurther cause the computer system to: identify geographic coordinates ofa portion of a first image in the first set of images using theindicated plant health; and transmit a message intended for anirrigation system corresponding to the geographic coordinates, whereinthe message includes an instruction to activate a sprinkler head in theirrigation system more frequently.